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Spatial Scalable Compression Scheme Using Spatial Sharpness Enhancement Techniques 



FIELD OF THE INVENTION 

The invention relates to a video encoder/decoder, and more particularly to a 
video encoder/decoder with spatial scalable compression schemes using spatial sharpness 
enhancement techniques. 

5 

BACKGROUND OF THE INVENTION 

Because of the massive amounts of data inherent in digital video, the 
transmission of full-motion, high-definition digital video signals is a significant problem in 
the development of high-definition television. More particularly, each digital image frame is 

10 a still image formed from an array of pixels according to the display resolution of a particular 
system. As a result, the amounts of raw digital information included in high-resolution video 
sequences are massive. In order to reduce the amount of data that must be sent, compression 
schemes are used to compress the data. Various video compression standards or processes 
have been established, including, MPEG-2, MPEG-4, and H.263. 

1 5 Many applications are enabled where video is available at various resolutions 

and/or qualities in one stream. Methods to accomplish this are loosely referred to as 
scalability techniques. There are three axes on which one can deploy scalability. The first is 
scalability on the time axis, often referred to as temporal scalability. Secondly, there is 
scalability on the quality axis, often referred to as signal-to-noise scalability or fine-grain 

20 scalability. The third axis is the resolution axis (number of pixels in image) often referred to 
as spatial scalability or layered coding. In layered coding, the bitstream is divided into two or 
more bitstreams, or layers. Each layer can be combined to form a single high quality signal. 
For example, the base layer may provide a lower quality video signal, while the enhancement 
layer provides additional information that can enhance the base layer image. 

25 In particular, spatial scalability can provide compatibility between different 

video standards or decoder capabilities. With spatial scalability, the base layer video may 
have a lower resolution than the input video sequence, in which case the enhancement layer 
carries information which can restore the resolution of the base layer to the input sequence 
level. 



WO 03/036980 



PCT7IB02/04298 



2 

Figure 1 illustrates a known layered video encoder 100. The depicted 
encoding system 100 accomplishes layer compression, whereby a portion of the channel is 
used for providing a low resolution base layer and the remaining portion is used for 
transmitting edge enhancement information, whereby the two signals maybe recombined to 
bring the system up to high-resolution. The high resolution video input is split by splitter 102 
whereby the data is sent to a low pass filter 104 and a subtraction circuit 106. The low pass 
filter 104 reduces the resolution of the video data, which is then fed to a base encoder 108. In 
general, low pass filters and encoders are well known in the art and are not described in detail 
herein for purposes of simplicity. The encoder 108 produces a lower resolution base stream 
which is provided to a second splitter 110 from where it is output from the system 100. The 
base stream can be broadcast, received and via a decoder, displayed as is, although the base 
stream does not provide a resolution which would be considered as high-definition. 

The other output of the splitter 110 is fed to a decoder 112 within the system 
100. From there, the decoded signal is fed into an interpolate and upsample circuit 114. In 
general, the interpolate and upsample circuit 1 14 reconstructs the filtered out resolution from 
the decoded video stream and provides a video data stream having the same resolution as the 
high-resolution input. However, because of the filtering and the losses resulting from the 
encoding and decoding, certain errors are present in the reconstructed stream. These errors 
are determined in the subtraction circuit 106 by subtracting the reconstructed high-resolution 
stream from the original, unmodified high-resolution stream. The output of the subtraction 
circuit 106 is fed to an enhancement encoder 116 which outputs a reasonable quality 
enhancement stream. 

The disadvantage of filtering and downscaling the input video to a lower 
resolution and then compressing it is that the video loses sharpness. This can to a certain 
degree be compensated for by using sharpness enhancement after the decoder. Although this 
can be made to work reasonably well for most parts of the video picture, there are some areas 
within the picture where the result remains poor compared to the original picture, e.g., small 
text parts will remain unreadable even with the most sophisticated enhancement. 

SUMMARY OF THE INVENTION 

The invention overcomes the deficiencies of other known layered compression 
schemes by increasing the video compression of a scalable compression scheme by the 
introduction of a number of video enhancement techniques on the base layer. Using a video 
picture analyzer, the best mix of the various video enhancement techniques is determined and 
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parameters defining this mix are transmitted to the decoder section as user data. The video 
picture analyzer compares the selected mix of enhanced bitstreams with the original foil 
resolution input signal and determines for which pixels a residual enhancement layer is 
required. 

According to one embodiment of the invention, a method and apparatus for 
encoding and decoding an input video bitstream is disclosed. A base bitstream and a residual 
bitstream are encoded in the following manner. A decoded upscaled base bitstream is 
enhanced in a first plurality of enhancement units having different enhancement algorithms 
and a plurality of enhanced base video streams are outputted. The input video bitstream is 
compared with the decoded upscaled base bitstream and the enhanced base video streams, 
where the output of the picture analyzer controls the information contained in the residual 
bitstream. The base bitstream and the residual bitstream are decoded in the following 
manner. The same enhancement is performed on the decoded base bitstream as was 
performed in the encoder unit. The decoded residual bitstream is superimposed on the 
decoded and enhanced base video stream to produce a video output bitstream. 

According to another embodiment of the invention, a mix of the enhanced 
base video streams and the decoded upscaled base bitstream can be used to control the 
information in the encoded residual bitstream, i.e., which pixels pr groups of pixels should be 
included in the decoded residual bitstream. 

These and other aspects of the invention will be apparent from and elucidated 
with reference to the embodiments described hereafter. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The invention will now be described, by way of example, with reference to the 
accompanying drawings, wherein: 

Figure 1 is a block diagram representing a known layered video encoder; 

Figure 2 is a block diagram of a layered video encoder/decoder according to 
one embodiment of the invention; 

Figure 3 is a block diagram of a layered video encoder/decoder according to 
one embodiment of the invention; and 

Figure 4 is an illustration of a vector candidate set location according to one 
embodiment of the invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

According to one embodiment of the invention, a spatial scalable compression 
scheme using spatial sharpness enhancement techniques is disclosed. Briefly, the filtered and 
downscaled video sequence is compressed. Then, out of the decoded base layer frames, 
several upscaled versions are processed using a variety of enhancement algorithms. This can 
include a standard upscaled and filtered, for example, nyquist filtered, versions as well as 
various shaipness enhancement algorithm versions. A picture analyzer processes all of the 
information and selects the best or the best mix of these versions. The mix parameters which 
define the selected mix is also inserted in the encoded residual bitstream, as user data, so that 
the decoder can exactly reproduce this enhancement. 

However, in some areas of the sharpness enhanced frames, the results will 
remain inadequate. By comparing in the encoder the available original full resolution frames 
with the enhancement frames, these areas can be detected. Only these detected areas will be 
compressed and be part of the residual bitstream which is inputted into the enhancement 
layer. The decoder then decodes the base layer downscaled bitstream and applies the same 
enhancement parameters on the decoded output as was performed in the encoder. The 
decoder then decodes the residual bitstream and superimposes the decoded bitstream on the 
pixels of the already decoded and enhanced base layer frames. 

This embodiment will now be described in more detail with reference to 
Figure 2 which is a block diagram of an encoder/decoder which can be used with the 
invention. The depicted encoding/decoding system 200 accomplishes layer compression, 
whereby a portion of the channel is used for providing a low resolution base layer and the 
remaining portion is used for transmitting edge enhancement information, whereby the two 
signals may be recombined to bring the system up to high-resolution. The high resolution 
video input 201 is split by a splitter 210 whereby the data is sent to a low pass filter 212, for 
example a nyquist filter, and a splitter 232. The low pass filter 210 reduces the resolution of 
the video data, which is then fed to a base encoder 214. In general, low pass filters and 
encoders are well known in the art and are not described in detail herein for purposes of 
simplicity. The base encoder 214 produces a lower resolution base stream 215. The base 
stream can be broadcasted, received and via a decoder, displayed as is, although the base 
stream does not provide a resolution which would be considered as high-definition. 

The encoder also outputs a decoded base stream to an upscaling circuit 216. 
In addition, a decoder (not illustrated) can be inserted into the circuit after the encoder 214 to 
decode the output of the encoder prior to being sent to the upscaling circuit 216. In general, 
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the upscaling circuit 216 reconstructs the filtered out resolution from the decoded video 
stream and provides a video data stream having the same resolution as the high-resolution 
input. The upscaled bitstream vl from the upscaling circuit 216 is split by a splitter 218 and 
inputted into a picture analyzer 230, a subtraction circuit 234 and a splitter 220. The 
upscaled bitstream vl from splitter 220 is inputted into enhancement units 222 and 224. 
Each enhancement unit operates a different spatial enhancement algorithm which will be 
explained in more detail below. Figure 2 has two enhancement units but it will be 
understood that any number of enhancement units can be used in the invention. 

Many video enhancement techniques exist and they all modify the picture 
content such that the appreciation of the resulting picture is improved. The subjective 
characteristic of these enhancements complicate the optimization process and is likely the 
reason for the diversity in video enhancement algorithms. Various enhancement algorithms 
contribute by some means to the picture quality, and as a result, they often appear in a chain 
to profit from the individual strengths. Noise reduction and sharpness improvement 
algorithms are just a few examples out of a large set of enhancement algorithms. It will be 
understood that any of these known enhancement algorithms can be used in the invention. 

. A high-quality spatial enhancement function consists of a collection of 
algorithms that contribute to different aspects of sharpness. Some algorithms only improve 
the gradients in the picture by increasing its steepness, whereas others modify the amplitude 
of the gradients. It may seem that these algorithms are mutually exclusive, however, this is 
far from true. Both means to improve the gradient characteristics may be used, where a 
predefined model determines the individual contribution of each algorithm. 

Returning to Figure 2, the upscaled bitstreams vl are processed in 
enhancement units 222 and 224 according to the enhancement algorithms in each unit The 
resulting video streams from enhancement units 222 and 224 are inputted into subtraction 
units 226 and 228 respectively, wherein the bitstream vl is subtracted from the resulting 
video streams from enhancement units 222 and 224 to produce video streams v2 and v3, 
respectively. Video streams v2 and v3 are inputted into the picture analyzer 230. The input 
bitstream 201 is also inputted into the picture analyzer 230 via splitter 232. The picture 
analyzer 230 compares vl , v2 and v3 with the original bitstream and determines how best to 
enhance the picture. The picture analysis performed by the picture analyzer can be 
performed in a variety of ways. For example, the picture analyzer 230 could compare vl, v2 
and v3 with the original picture and select the video stream (vl, v2 or v3) which best 
approximates the original picture. Alternatively, the picture analyzer can use a mix of the 
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different bitstreams using mix parameters (a, j8) or enhancement vectors such that the 
optimum overall picture quality is achieved from a combination of video streams. For 
example, the picture analyzer can select a vector representing the mix parameters for 
calculating the mixture of the enhanced base video streams to control the information in the 
residual bitstream using the selected vector. Furthermore, bit cost function can also be used 
in determining the best mix parameters as will be explained below with reference to Figure 3. 
It will be understood that other schemes than the ones described can be used in the picture 
analyzer 230 and the invention is not limited thereto. 

There are numerous advantages to using mix parameters in the picture 
analyzer 230. Firstly, this is a completely expandable system. If there are more functions to 
contribute to the sharpness of the picture, they can be easily accounted for. The new 
functions need not be optimized for the system. Secondly, the interdependencies of various 
functions can be accounted for while deciding on the suitable enhancement vectors. Thirdly, 
a spatio-temporal consistency model can be incorporated in the picture analyzer 230. 

The upscaled output of the upscaling circuit 216 is subtracted from the original 
input 201 in a subtraction circuit 234 to produce a residual bitstream which is applied to a 
switch 236. The switch is controlled by the output of the picture analyzer 230. By 
comparing the input video bitstream 201 with the various enhanced base video streams, the 
picture analyzer 230 can determine which pixels or groups of pixels (blocks) need to be 
further enhanced by the enhancement layer 208. For the pixels or groups of pixels (blocks) 
that are determined to need enhancement by the picture analyzer 230, the picture analyzer 
230 outputs a control signal to close switch 236 to let those parts of the residual bitstream 
through to the enhancement layer encoder 240. The picture analyzer 230 also sends the 
selected mix parameters and the control signal for the switch to the encoder 240 so that this 
information is encoded with the resulting residual bitstream from switch 236 and outputted as 
the enhancement stream 241. 

The base stream 215 is sent to a base decoder 250 and the enhancement stream 
241 is sent to an enhancement encoder 252 in the decoder section 204. The decoder 250 
decodes the base stream 215 which is then upscaled by an upscaling circuit 254. The 
upscaled decoded bitstream is then split by a splitter 256 and sent to enhancement units 262 
and 264, merge unit 270 and addition unit 272. Enhancement unit 262 comprises the same 
spatial enhancement algorithm as enhancement unit 222 and enhancement unit 264 comprises 
the same spatial enhancement algorithm as enhancement unit 224. The enhancement units 
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262 and 264 perform their respective algorithms and send outputs v2 and v3 to the merge unit 
270. 

The enhancement decoder 252 decodes the enhancement stream and outputs 
the residual bitstream to the addition unit 272. In addition, the decoder 252 decodes the mix 
parameters and control signal and send this information to the merge unit 270. The merge 
unit merges together all of the inputs to create the enhancement output from the picture 
analyzer 230. The upscaled decoded base stream and the decoded residual bitstream are 
combined together by the addition unit 272 and the resulting bitstream is applied to the 
switch 274. The switch 274 is controlled by the control signal so that the output of the merge 
unit 270 can be applied to the appropriate pixels or blocks in the bitstream outputted by the 
addition unit 272 so as to produce the output signal 276. 

Figure 3 is a block diagram of an encoder/decoder 300 according to another 
embodiment of the invention. Many of the components in Figure 3 are the same as the 
components illustrated in Figure 2 so they have been given the same reference numerals. In 
addition, for the sake of brevity, the operations of the similar components will not be 
described. In this embodiment, a cost function is used in determining the mix parameters a, 
0,... for the individual enhancement signals v2, v3, . . . According to one embodiment of the 
invention, enhancement vectors are assigned on a block by block basis. Previously 
determined best enhancement vectors from a spatio-temporal neighborhood are evaluated in a 
cost function as illustrated in Figure 4. The cost function calculates a metric mat is related to 
the objective picture quality. The best estimate of the enhancement vector is defined by one 
yielding the smallest cost, i.e., Best vector = min e(Oi, ft ...) where i = 1 , 2, . . . number of 
candidates and e() is the cost function with vectors <&&... as parameters. 

The cost function should incorporate within itself all the factors that define 
good quality and also artifact prevention mechanism. For example, in case of sharpness 
enhancement function, the steepness of the gradients is an important factor and should be 
accounted for in the cost function. Artifacts like aliasing that result from sharpness 
improvement should also be included in Ihe cost function. The cost function serves as a 
quality measure. 

Returning to Figure 3, the enhancement layer encoder 240 sends bitcost 
information to the picture analyzer 230. The cost function is calculated from the mixed 
signal Ve(a,j3,. . .) for a limited set of parameters og8,. . . The better the picture quality of the 
signal Ve(ce,/3,. . .), the lower the cost function becomes. For every pixel or group of pixels a 
few vectors of ajS,. . . can be tested. The test vector with the lowest cost function is then 
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selected. In one embodiment, some of the test vectors are already selected vectors of 
neighboring, in time and space (previous frame), group of pixels. For example, vectors \fi u 
htfa, X 3 /3 3 illustrated in Figure 4 are neighbors of the group of pixels being tested. In addition 
one or more vectors can be selected with a random offset. The picture analyzer outputs Ve = 
vl+av2+/3v3 and Vb=vl+Av2+/*v3 where a, (8 are the mix parameters and \ H are the cost 
function parameters. In this embodiment, the signal Vb is subtracted from the original input 
bitstream in the subtracter 234 to form the residual bitstream. Whenever the final cost 
function exceeds a predetermined threshold limit, the picture analyzer outputs a signal s to 
the switch 236 so that the switch will close and for that group of pixels a residual bitstream is 
encoded in the encoder 240. In addition, the picture analyzer also sends the control signal, 
the mix parameters and the cost function to the encoder 240 which are then coded and 
inserted into the enhancement stream 241 . When the enhancement stream is decoded in the 
enhancement decoder 252, the mix parameters and cost function are decoded and sent to the 
merge unit 270. The merge unit outputs Vb which is added to the decoded enhancement 
stream in the addition unit 272 and the resulting bitstream is applied to the switch 274. The 
switch 274 is controlled by the control signal S so that Ve from the merge unit 270 can be 
applied to the appropriate pixels or blocks in the bitstream outputted by the addition unit 272 
so as to produce the output signal 276. 

The above-described embodiments of the invention enhance the efficiency of 
spatial scalable compression by using a picture analyzer to select the best or a mix of a 
plurality of enhanced base bitstreams via determined enhancement vectors to control the 
information in the encoded residual bitstream. It will be understood that the different 
embodiments of the invention are not limited to the exact order of the above-described steps 
as the timing of some steps can be interchanged without affecting the overall operation of the 
invention. Furthermore, the term "comprising" does not exclude other elements or steps, the 
terms "a" and "an" do not exclude a plurality and a single processor or other unit may fulfill 
the functions of several of the units or circuits recited in the claims. 
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CLAIMS: 



1 # a layered encoder for encoding an input video bitstream, the encoder 

comprising: 

a layered encoder unit for encoding a base bitstream at a lower resolution and 
a residual bitstream, the layered encoder unit comprising: 
5 a number of enhancement units, each with a different enhancement 

algorithm for enhancing a decoded upscaled base stream and outputting enhanced base video 
streams; 

a picture analyzer for comparing the input video bitstream with the 
decoded upscaled base bitstream and the enhanced base video streams, where the output of 
10 the picture analyzer controls the information included in the residual bitstream. 

2. The layered encoder according to claim 1 , wherein the picture analyzer selects 
a vector representing mix parameters for calculating the mixture of the enhanced base video 
streams and controls the information in the residual bitstream using the selected vector. 

3. The layered encoder according to claim 2, wherein the picture analyzer 
compares the selected mixture of enhanced base video streams with the input video bitstream 
to determine for which pixels or group of pixels additional enhancement is required via the 
residual bitstream. 

4 # The layered encoder according to claim 2, wherein each group of pixels is 

enhanced using different vectors. 

5 The layered encoder according to claim 2, wherein the picture analyzer 

25 calculates a cost function for a limited number of test vectors and the test vector with the 
lowest cost function is selected. 



15 



20 



6 The layered encoder according to claim 5, wherein the selected vector is 

included in a compressed data stream. 
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7 The layered encoder according to claim 5, wherein a number of the test vectors 

are already selected vectors of neighboring, in time and space, group of pixels. 

The layered encoder according to claim 1, wherein the layered encoding unit 
further comprises: 

a downsampling unit for reducing the resolution of the input video bitstream; 

a base encoder for encoding the lower resolution base stream; 

an upscaling unit for decoding and increasing the resolution of the base stream 
to produce an upscaled base bitstream; 

a subtraction unit for subtracting the upscaled base bitstream from the input 
video bitstream to produce the residual bitstream; 

switching means for selectively allowing only portions of the residual 
bitstream to be sent to an enhancement encoder based upon a control signal from the picture 
analyzer; 

the enhancement encoder for encoding the portions of the residual bitstream 
which pass through the switching means to form the encoded residual bitstream. 

9 The layered encoder according to claim 8, wherein said switching means is a 
multiplier having a value between 0 and 1, wherein a value of 0 means the switching means 
is open and a value of 1 means the switching means is closed. 

10 a layered decoder unit for decoding a base bitstream and a residual bitstream, 
the layered decoder unit comprising: 

means for enhancing the decoded base bitstream, the means for enhancing 
comprising a plurality of enhancement units having different enhancement algorithms for 
outputting an enhanced base video stream, and 

means for superimposing the decoded residual bitstream on the enhanced base 

video stream. 

11. a layered decoder unit as claimed in claim 10, wherein the decoder is arranged 

to receive a vector representing mix parameters for calculating the mixture of enhanced base 
streams produced by the plurality of enhancement units in order to produce the enhanced 
base video stream. 
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12. A method for encoding an input video bitstream the method comprising the 
steps of: 

encoding a base bitstream and a residual bitstream, comprising the steps of: 
enhancing a decoded upscaled base bitstream in a plurality of 
different enhancement algorithms outputting enhanced base video streams; 

comparing the input video bitstream with the decoded upscaled base 
bitstream and the enhanced base video streams, where the output of the comparison controls 
the information contained in the residual bitstream. 

1 3 . The method according to claim 1 2, wherein a vector representing mix 
parameters for calculating a mixture of the enhanced base video streams is selected controls 
the information in the residual bitstream using the selected vector. 

14. The method according to claim 13, wherein the selected mixture of enhanced 
base video streams is compared with the input video bitstream to determine for which pixels 
or group of pixels additional enhancement is required via the residual bitstream. 

15. The method according to claim 13, wherein each group of pixels is enhanced 
using different vectors. 

16. The method according to claim 13, wherein a cost function for a limited 
number of test vectors is calculated and the test vector with the lowest cost function is 
selected. 

17. The method according to claim 16, wherein the selected vector is included in a 
compressed data stream. 

18. The method according to claim 16, wherein a number of the test vectors are 
already selected vectors of neighboring, in time and space, group of pixels. 

19. The method according to claim 12, further comprising the steps of: 
reducing the resolution of the input video bitstream; 

encoding the lower resolution base stream; 
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decoding and increasing the resolution of the base stream to produce an 

upscaled base bitstream; 

subtracting the upscaled base bitstream from the input video bitstream to 

produce the residual bitstream; 

selectively allowing only portions of the residual bitstream to be sent to an 
enhancement encoder based upon a control signal from the picture analyzer; 

encoding the selectively allowed portions of the residual bitstream to form the 
encoded residual bitstream. 

20. A method of decoding a base bitstream and a residual bitstream, the decoding 
comprising: 

enhancing the decoded base bitstream in a plurality of different enhancement 
algorithms for outputting an enhanced base video stream, and 

superimposing the decoded residual bitstream on the enhanced base video 

stream. 

21 . A method of decoding as claimed in claim 20, wherein the method further 
comprises receiving a vector representing mix parameters for calculating the mixture of 
enhanced base streams produced by the plurality of enhancement units in order to produce 
the enhanced base video stream. 

22. A compressed data stream including: 

a base bitstream and a residual bitstream, wherein the information included in 
the residual bitstream represents a difference between a bitstream at higher resolution than 
the base bitstream and an enhanced decoded upscaled base bistream, which enhanced 
decoded upscaled base bitstream is based on the base bitstream and wherein the enhancement 
has been performed by a mixture of a plurality of enhancement algorithms. 

23. A compressed data stream as claimed in claim 22, wherein the compressed 
data stream includes a vector representing mix parameters for calculating the mixture. 



24 a storage medium on which a compressed data stream as claimed in claim 22 

or 23 has been stored. 
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25. A layered encoder/decoder for encoding and decoding an input video 

bitstream, comprising: 

a layered encoder unit for encoding a base bitstream at a lower resolution and 
a residual bitstream, the layered encoder unit comprising: 

a number of enhancement units, each with a different enhancement 
algorithm for enhancing a decoded upscaled base stream and outputting enhanced base video 
streams; 

a picture analyzer for comparing the input video bitstream with the 
decoded upscaled base bitstream and the enhanced base video streams, where the output of 
the picture analyzer controls the information contained in the residual bitstream; 

a layered decoder unit for decoding the base bitstream and the residual 
bitstream, the layer decoder unit comprising: 

means for performing the same enhancement to the decoded base 
bitstream as was performed in the encoder unit; and 

means for superimposing the decoded residual bitstream on the 
decoded and enhanced video base stream to produce a video output stream. 

26. A computer program product comprising software code portions for 
performing the steps of claim 12 when said product is run on a computer. 

27. A computer program product comprising software code portions for 
performing the steps of claim 20 when said product is run on a computer. 



WO 03/036980 



PCT/IB02/04298 




WO 03/036980 



PCT/IB02/04298 



200 



2/4 

^ 2 >T > 0 sl 6 enhencoder 208 

S 



210 

A. 



212 



INPUT 
201 



BASE-ENCODER 
214 







enc 


1 





nyquist 



r 



216 



222 

». 



224 



tt 



Vb 



enc 



. -240 

-t to Mux 



enh stream 
241 



picture 
analyser 



1 



v1. 



a 



~9 



220 
v2 



226 



tt 



+ A v3 



7 

228 



-230 



Ve=v1 +<XV2+0V3 

base stream 
215 



nyquist 

spatial enhancement algo 1 
spatial enhancement algo 2 



spatial enhancement algo 2 
spatial enhancement algo 1 

nyquist 



v1+av2+jSv3 



OUTPUT- 



v3 I 



0 



v2 
256 



tt 



-264 



♦ ! r 



V1 

1 



merge 



276 



o 
274 



tt 



262 



-254 



250- 



dec 



base stream 



BASE-DECODER 



+ 

9^ 



272 

FIG. 2 



dec 



enh stream 



—252 



ENH-DECODER 



WO 03/036980 



PCT/IB02/04298 



300 



3/4 

232 234 236 ENH-ENCODER 208 
s 



212 



210 



INPUT 
201 



Vb=v1+A.v2+/zy3- 

BASE-ENCODER 
-214 



enc 

h -240 



picture 
analyser 









1 





enc 



nyquist 



216 , 
v1 

"^218 



222 

► 



224 

* 



tt 



tf 



n 

226 
+ 



220 
v2 



228 



v3 



enh stream 
— 241 



-230 



Ve=v1+av2+j8v3 



base stream 
— 215 



nyquist 

spatial enhancement algo 1 
spatial enhancement algo 2 



spatial enhancement algo 2 
spatial enhancement algo 1 

nyquist 



v1+otv2+j8v3 



v3 



-e- 



P 



v2 



tt 



-264 



a 



v1 
1 



merge 



270-^ 



OUTPUT 



276 L 



tt 



-262 



254 



250 

1 



dec 



base stream 



BASE-DECODER 



272 

FIG. 3 



v1 +Av2+/^y3 

9^ 



dec 



enh stream 



-252 



ENH-DECODER 



WO 03/036980 



4/4 



PCT/IB02/04298 



n-1 











\ 

\ 

s. 














<x3jff3 




y 


















V 


X 










\ 

\ 

n v 




a 1,01 


■ 













FIG. 4 



INTERNATIONAL SEARCH REPORT 



lnternatlo| plication No 

PCT/IB 02/04298 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC 7 H04N7/26 H04N7/30 H04N7/50 



According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 

IPC 7 H04N 



Documentation searched other than minimum documentation to the extent that such documents are Included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practical, search terms used) 

EPO-Internal 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category ° Citation of document, with indication, where appropriate, of the relevant passages 



US 6 128 041 A (KIM JIN-HUN 
3 October 2000 (2000-10-03) 
page 4, line 38 - line 56 



ET AL) 



column 7, line 14 - line 48 

column 9, line 51 -column 11, line 39 

US 6 011 872 A (QIAN RICHARD J ET AL) 

4 January 2000 (2000-01-04) 

column 2, line 28 - line 39 

column 5, line 38 -column 11, line 13 

US 2001/024470 Al (CHEN YINGWEI ET AL) 
27 September 2001 (2001-09-27) 

paragraph '0039! - paragraph '0040! 

paragraph '0043! - paragraph '0046! 

-/-- 



Relevant to claim No. 



1,10,12, 
25 

2,5,6,8, 
22 



1,10,12 



1,10,12 



no 



Further documents are listed in the continuation of box C. 



El 



Patent family members are listed in annex. 



0 Special categories of cited documents : 

•A' document defining the general state of the art which is not 

considered to be of particular relevance 
a E" earlier document but published on or after the International 

filing date 

'L' document which may throw doubts on priority clalm(s) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

"O" document referring to an oral disclosure, use, exhibition or 
other means 

"P" document published prior to the international filing date but 
later than the priority date claimed 



■T* later document published after the International filing date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
Invention 

"X" document of particular relevance; the claimed invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document is taken alone 

"Y a document of particular relevance; the claimed invention 

cannot be considered to involve an inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the art. 

'&" document member of the same patent family 



Date of the actual completion of the International search 



14 January 2003 



Date of mailing of the international search report 

03/02/2003 



Name and mailing address of the ISA 

European Patent Office, P.B. 5818 Patentlaan 2 
NL - 2280 HV Rljswqk 
Tel. (+31-70) 340-2040, Tx. 31 651 epo nl, 
Fax: (+31-70) 340-3016 



Authorized officer 



Marie- Julie, J-M 



Foim PCT/lSA/210 (saoond sheet) (Jufy ig©2) 



page 1 of 2 



INTERNATIONAL SEARCH REPORT 



Internatfcj pplication No 

PCT/IB U2/04298 



C.(Contlnuation) DOCUMENTS CONSIDERED TO BE RELEVANT 



Category ° CBation of document, with Indication t where appropriate, of the relevant passages 



Relevant to claim No. 



WO 99 33274 A (ROSE KENNETH) 

1 July 1999 (1999-07-01) 

page 8, line 16 -page 10, line 1 

US 5 828 788 A (ZDEPSKI JOEL WALTER 
AL) 27 October 1998 (1998-10-27) 
column 4, line 45 - line 64 
column 5, line 30 - line 52 
column 7, line 29 - line 59 



ET 



1-27 



1-27 



Form PCT/lSA/210 (continuation of second sheet) (July 1992) 



page 2 of 



2 



INTERNATIONAL SEARCH REPORT 

tntB^pitton on patent family members 



InternatioJ plication No 

PCT/IB 02/04298 



. Patent document 




Publication 




Patent family 


Publication 


cited in search report 




date 




member(s) 


date 


US 6128041 


A 


03-10-2000 


DE 


19739266 Al 


14-01-1999 








GB 


2327310 A ,B 


20-01-1999 








ID 

Jr 


llUbbOM A 


o£ no i Ann 


1 IC Am 1 Q70 

ub oUilo/^ 


A 

A 


U4— Ul-ZUUU 


CD 

tr 


1 rtn 0700 A 1 

lOlZ/oo Al 


28-Oo-ZOOO 








WO 


9821694 Al 


22-05-1998 








JP 


2001504296 T 


27-03-2001 


US 2001024470 


Al 


27-09-2001 


US 


6292512 Bl 


18-09-2001 








EP 


1040668 A2 


04-10-2000 








WO 


0002392 A2 


13-01-2000 








JP 


2002520920 T 


09-07-2002 


WO 99332/4 


A 


01-07-1999 


All 

AU 


1928999 A 


12-07-1999 








WO 


9933274 Al 


ai r>~7 i nor* 

01-07-1999 


US 5828788 


A 


27-10-1998 


All 

AU 


6315396 A 


30-01-1997 








DE 


69605117 Dl 


16-12-1999 








DE 


69605117 T2 


25-05-2000 j 








EP 


0835590 Al 


1 f% m i Ann 

15-04-1998 i 








wo 


9701935 Al 


16-01-1997 








JP 


11508738 T 


27-07-1999 








AU 


713904 B2 


16-12-1999 








AU 


5908596 A 


30-01-1997 








BR 


9609462 A 


27-07-1999 








EP 


0835589 Al 


15-04-1998 








WO 


9701934 Al 


16-01-1997 








JP 


11508737 T 


27-07-1999 



Foim PCT/ISA/210 (patent family annex) (July 1092) 



